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Civic Education in the United States 



In the United States, civic educators are debating the need to mandate the 
inclusion of civic education in high school curriculum. The renewed interest 
in civic education stems from a decline in the political involvement of youth 
over the previous two decades: only about one-third of young people aged 
18-29 voted in the 2000 presidential election, compared with one-half that did 
so in the 1970s. 1 Surveys and behavioral studies conducted over the past few 
years routinely found high levels of apathy, low rates of voter turnout, a loss 
of confidence in governmental institutions, and poor showings on history and 
civics tests. 2 On the 1998 National Assessment of Educational Progress (NAEP) 
civics assessment, only 26 percent of a nationally representative sample of 
twelfth-graders scored at the proficient level or higher. 3 

The decline of political engagement among youth has sparked an increase 
in research on civic education. During the past 10 years, researchers have 
conducted numerous school-based studies of civic education to assess the role 
of civic education classes and their effect on students. Although these studies 
yielded some interesting findings, many had methodological limitations, 
making generalizations based upon the studies problematic. Limitations 
include: small sample size; no random assignment or representative sample of 
classrooms or students; exclusion of attitudinal measures; and the inability to 
track students from pre-test to post-test. 4 

Because of these limitations, further research into whether civic education 
can increase political interest and participation in young people is needed. 
However, information on civic education in general is only one aspect of the 
research needed. The What Works Clearinghouse at the Institute of Education 
Sciences (IES) in the U.S. Department of Education has begun to focus on 
identifying specific curricula that can improve the quality of education in the 
United States. IES encourages studies that include experimental or high-quality 
quasi-experimental design because studies with methodological rigor provide 
the best opportunity to determine what works in education. 

Experimental and high-quality quasi-experimental studies require extensive 
resources; before conducting such studies, researchers gather as much 
information as possible about the potential impact of an intervention or 
curriculum. By conducting a pilot study in which a small number of students 
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are exposed to an intervention, researchers can measure the effectiveness 
of their instruments, such as surveys or assessments, and gauge the relative 
impacts on treatment and comparison groups. This information helps ensure 
that the full-scale study is conducted with the utmost efficiency and rigor, as 
well as guarantee effective use of resources. This report describes the result of a 
pilot test for one curriculum, We the People: the Citizen and the Constitution. 
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The Curriculum 

We the People: the Citizen and the Constitution is published by the Center 
for Civic Education, located in Calabasas, California. It is a program that 
teaches students about American constitutional democracy, with the primary 
goal of promoting civic competence and responsibility among the nation’s 
elementary, middle, and secondary students. Funded by Congress, the program 
is taught in all 50 states. 

The instructional program is intended to enhance students' 
understanding of the institutions of American constitutional 
democracy and to reinforce the contemporary relevance of 
the Constitution and the Bill of Rights. The culminating 
activity is a simulated congressional hearing in which students 
"testify" before a panel of judges. Students demonstrate their 
knowledge and understanding of constitutional principles and 
evaluate, take, and defend positions on relevant historical and 
contemporary issues. 

Since the inception of the We the People program in 1987, 
more than 26 million students and 82,000 educators have 
participated in the program. We the People enjoys the active 
participation of members of Congress, as well as support from 
professional, business, and community organizations across 
the nation. 

Background of Study 

In January 2003, the Center for Civic Education contracted with MPR 
Associates, Inc. to evaluate the We the People civic education curriculum. 

The evaluation included an assessment of student knowledge of civics and 
understanding of democratic principles and practices, as well as a measure of 
change in student attitudes towards civic and political participation. In Fall 
2003, MPR conducted a pilot test of the knowledge assessment and survey 
instruments. This report presents the findings and results of that pilot study. 

The Pilot Test 

The pilot test had two components: 

■ A survey of student attitudes towards civic and political participation. 
This survey was administered at the beginning and again at the end of 
the course. 

■ An assessment of student knowledge about U.S. government and 
political history. The knowledge assessment was administered at the 
end of the course. 




Students who took courses using the We the People curriculum were compared 
with students who took courses using other civics curricula. 
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The Survey: Design and Validity 

The survey of student attitudes towards civic and political participation was 
designed by MPR Associates, Inc. in collaboration with Dr. Richard Niemi at 
the University of Rochester and Dr. Elizabeth Theiss-Morse at the University 
of Nebraska — Lincoln, both eminent scholars of political attitudes and beliefs. 
The survey consisted of 85 questions and measured the extent to which 
students engaged in various types of political participation and their attitudes 
towards government and citizens' responsibilities. 

The survey drew on previous attitude surveys that have been successful with 
high school students and that were found to be statistically valid and reliable. 
In addition, the survey underwent scrutiny by a focus group of high school 
seniors before being pilot-tested in Fall 2003. 

The survey was constructed around a series of questions, called "item banks," 
that gauge student attitudes or feelings towards a single concept about politics 
and the community. Answers for each item within the item bank were then 
aggregated to a single "score" for each student for that item bank. In some 
cases, certain items were not included in the total item bank score. In general, 
an item was eliminated when it did not fit, or "scale" well with the other items 
in the bank. In the description of each category below, items not included in 
the total score are indicated. The categories of questions asked on the survey 
included: 

Attention to politics. Do students pay attention to politics on television 
or radio, in the newspaper or news magazines, or on the Internet? Do 
they discuss politics with friends or family? 

Use of information. Do students analyze public policies, draw 
information from multiple sources, listen to opinions of others, and 
follow current events, both in and outside of class? 

Political participation. Have students worked for a political party or 
candidate, worn a button or displayed a bumper sticker, contacted a 
public official or the media about a political issue or cause, participated 
in a peaceful demonstration or protest, or boycotted or "buy-cotted" 
products? 

Political efficacy. Do students feel that they could make a difference 
in the decisions made by the government? Did they feel qualified to 
participate in or to lead political bodies? Do they feel well informed 
about politics? 
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Assessment of governments' unpact on their lives. Do students believe 
that federal, state, or local governments have a lot, some, a little, or no 
impact on their own lives? 



We the People 



Evaluation of government. How do students describe their feelings 
(negative or positive) towards the U.S. Presidency as an institution, the 
U.S. Congress, the U.S. Supreme Court, and the U.S. Constitution? 

Appreciation of democracy. What do students believe about 
representative government, deliberation in the legislature, the use 
of experts to resolve issues, and the potential benefits or problems 
associated with direct democracy? 

Citizens' responsibilities. How important do students feel it is to vote, 
attend community meetings, work on campaigns, join a political group, 
or protest peacefully against laws they think unjust? 

Citizens' obligations. How essential it is to students to serve in the U.S. 
military (not included in factor score), give money to charities, pay taxes, 
obey laws, help when there is a disaster in the community, volunteer, or 
work hard at their jobs? 

Political tolerance. Do students believe that members of a group they 
disliked should be allowed to run for public office, teach in public 
schools, make public speeches, hold peaceful public rallies, have their 
phones tapped by the government, or be outlawed? 

Tolerance of ambiguity. Do students avoid looking at problems from 
multiple perspectives or at problems without one "best" solution? Do 
they enjoy tackling complex problems where the solution is not clear? 

Do they find complex problems interesting? 

Community responsibility. How important is it for students to be 
active in public affairs, well-off financially (not included in factor 
score), become a community leader, help others who are in difficulty, or 
contribute to the public good? 

At the conclusion of the course, students were also asked if they enjoyed 
the course, if they thought it made them more or less interested in politics 
and current events, and if they thought it made them more or less likely to 
participate in politics in the future. 

The results of the pilot test indicate that the attitudinal survey was very 
reliable and that the item banks were successful in measuring student attitudes 
in a consistent and cumulative fashion. Full results of the item bank reliability 
analyses are presented at the end of this report. 

Student responses to each of the items within a group were summed to reach 
an overall score for each student in each category. Scores were calculated for 
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each student on the survey at the beginning and again at the end of the term. 
A comparison between the pre-test and post-test survey scores was the basis for 
the statistical analyses. 

The Test: Design and Validity 

WestEd, a leading publisher of educational assessments, designed an 
assessment of student knowledge about politics and U.S. government. WestEd 
assembled a team of civics and government teachers and other experts in civic 
education who wrote, revised, and discussed a large number of potential items 
for the test. Eventually an item bank of more than 100 items was developed. 
The resulting assessment was used to measure student knowledge during this 
pilot test phase. 

The knowledge assessment included many central concepts of the five content 
areas assessed by the National Center for Education Statistics as part of their 
National Assessment of Educational Progress. They included the following: 

■ Civic life, politics and government: 

- Classical liberalism, and classical republicanism; 

- Personal, political, and economic rights; and 

- Evolution of ideas: individual rights, constitutionalism, 
secular government, and representative government. 

■ The foundations of the American political system: 

- Classical liberalism: political equality, unalienable rights, 
government's role in the protection of those rights, popular 
sovereignty, and the right to revolution; and 

- Classical republicanism: civic virtue, the common good, 
representative government, and moral education. 

■ The U.S. Constitution and its effects on the purposes, values, and 
principles of American democracy: 

- Framing the Constitution: issues, compromises, and the 
ratification process; 

- How the system is designed and works: institutions, processes, 
and locus of sovereignty; 

- Historical foundations of the Constitution: underlying values 
and principles, understanding of classical political history, 
(including Greek and Roman history, the British and colonial 
experience, and experience with state constitutions; 

- Development and expansion of rights: British and colonial 
antecedents, constitutional rights to freedom of belief, freedom 
of expression, due process of law, and equal protection; and 

- Statutory rights and court decisions, scope and limits of rights, 
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the incorporation of the 14th amendment, and the role of 
citizens in the expansion of rights. 

■ The relationship of the United States to other nations and to world 
affairs: 

- Comparative political systems: confederal, federal, parliamentary, 
and legal (both adversarial and inquisitorial); and 

- International political arrangements: the United Nations and the 
Universal Declaration of Human Rights. 

■ The roles of citizens in American democracy: 

- Personal and civic responsibility (obedience to laws, civic 
participation, jury service, military service, voting, etc.); and 

- Issues of individual rights versus the common good, political 
participation, and liberty versus equality. 

The test was piloted in Fall 2003. Results showed that the test is not easy 
and, therefore, should be a good measure of student attainment. A difficult 
test reduces the problems associated with ceiling effects, which occur when 
many high-performing students score so high that growth or differences in 
achievement cannot be documented. Reducing ceiling effects is particularly 
important when a curriculum is challenging and used with high-achieving 
students who tend to do well on tests. However, the test does not appear to 
be too difficult for average students and is therefore relevant to all students 
assessed and able to show growth and clear differences between groups where 
differences exist. 

The Method 

The primary goal of the pilot test was to provide enough information to 
determine the validity and reliability of the knowledge assessment and 
attitude survey instruments. A secondary but still important goal was to use 
information about student outcomes on the two instruments to permit an 
accurate power analysis to determine the necessary sample size for a future, 
full-scale study. 

Pilot Test Design 

The pilot test was designed as a comparison group analysis. Although surveys 
or knowledge assessments of young people can be very interesting, they 
can tell us little about the effects of the We the People curriculum without a 
comparison group. To help us understand how much young people change 
in terms of their knowledge and attitudes towards government and civic 
participation, we developed a group of comparison classrooms. In these 
classrooms, students participated in a civic education curriculum that also 
meets state standards, but that was different from We the People. The students 
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who participated in We the People in their civics class are called the "treatment 
group,” and those who did not are called the "comparison group." 

Students taking Advanced Placement (AP) Civics or Government often receive 
more intensive instruction than those in a normal civics class. Since We the 
People is used in AP and general education civics classes, it was important to 
obtain results for both types of courses. The final evaluation design used a 
four-cell comparison matrix. The numbers of students who fit into each cell 
are shown in the table below. 



Curriculum 


AP Course 


General Education 


Used We the People 
(treatment classes) 


257 students 


241 students 


Did not use We the People 
(comparison classes) 


1 60 students 


1 36 students 



Two sets of comparisons were made. The 257 students in AP courses using We 
the People were compared with the 160 students in AP courses not using We the 
People. Similarly, the 241 students in general education courses using We the 
People were compared with the 136 students in general education courses not 
using We the People. Finally, a series of regression models were used to control 
for factors such as parent educational level, ethnicity and race, gender, and 
other student characteristics that can affect test scores and survey responses. 

Sample of Schools 

For any study of a curriculum, it is important to ensure that the schools 
participating in the study are implementing the curriculum as it was intended. 
In the case of We the People, this means that the schools needed to use most of 
the textbook and to conduct simulated congressional hearings at the end of 
the term. Although, ideally, classroom observations can determine whether a 
curriculum is implemented fully, this is very expensive and time-consuming. 
For the pilot study, therefore, the most efficient way to ensure that a school 
implemented the curriculum properly was to include only schools that 
participated in regional We the People competitions in 2002-03. Therefore, 
all 16 districts (plus one additional private school) in California with schools 
that had participated in the prior year's regional competitions were originally 
considered for inclusion in the pilot study. The original set of 16 districts 
was eventually narrowed to seven participating districts. Within the seven 
remaining districts, 16 schools agreed to participate in the study and within 
them, 33 classrooms participated in all three data collections. 

Despite the limitations of the sampling method, the schools are quite 
representative of California high schools geographically and demographically. 
Two schools were in southern California, six were in the San Francisco/ 
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Sacramento region, and eight were in the Central Valley. The percentage of 
Hispanic students ranged from 5 percent to over 70 percent. The percentage 
of students receiving federal free/reduced-price lunches ranged from less than 
1 percent to 66 percent. The percentage of parents who had graduated from 
college ranged from 10 percent to over 80 percent. Some schools scored in the 
lowest 10 percent of high schools in the state, others in the top 1 percent, with 
most schools falling somewhere in between. 

Within each district, some schools used the We the People curriculum and 
others did not. To eliminate the influence of school-level factors as much 
as possible, schools were asked to contribute both a comparison class and a 
treatment class to the study. In most cases, schools could comply. In other 
cases, however, due to limited enrollment or other factors, we were unable to 
obtain both a comparison and treatment class from the same school. For this 
reason, some school-level indicators were included in the statistical analysis. 



Pilot Test Results 

Attitude Survey 

The attitude survey was analyzed by comparing student responses to questions 
on the survey at the beginning and end of the term. Some students showed 
a great deal of change over time, but others showed little change. The results 
were compared for students participating in We the People with those not 
participating. 
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The analysis took the following characteristics of students into consideration: 

■ Enrollment in an AP course 

■ Parent education level (if one parent had graduated from college) 

■ Whether the student liked the course 

■ Prior involvement in political or community groups 

■ Ethnicity 

■ Achievement in school (grades, highest level of math course 
completed) 

Controlling for these variables by using a selection model regression analysis, 
we found that students in courses using We the People were more likely to show 
greater growth in the following areas: 

■ Attention to politics 

■ Use of information 

■ Political participation 

■ Sense of political efficacy 

■ Responsibilities of a citizen 

■ Obligations of a citizen 

■ Sense of political and community responsibility 

For the following categories, there was no difference over time between 
students in classes using We the People and those not using that curriculum: 

■ Assessment of government's impact on their lives 

■ Evaluation of government 

■ Appreciation of democracy 

■ Political tolerance 

■ Tolerance of ambiguity 

In the categories where no difference was found between We the People 
students and the comparison group, four of the five categories were based 
upon an item bank with only a few (four or less) items. Two categories 
(appreciation of democracy and tolerance of ambiguity) had relatively low 
reliability scores (see Technical Tables). In addition, few of these categories are 
correlated with future political engagement . 5 

The seven categories where We the People students showed greater growth 
than the comparison group tend to be more directly associated with future 
political engagement. In particular, the sense of political efficacy and beliefs 
in the importance of citizens’ responsibilities and obligations are most closely 
associated with increased political engagement . 6 
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Knowledge Assessment 

The knowledge assessment was given near the end of the course. The average 
scale scores for each group of students are shown below. 

Comparison of Average Scale Score on Knowledge Assessment 

Percent 
100 

80 

60 

40 

20 

0 

Advanced Placement class Regular class 

■ We the People Other civics curriculum 

The scores show statistically significant differences between those students 
who used We the People and those students who used a different civics 
curriculum and between students enrolled in AP classes and students in regular 
classes. Additionally, there is a significant interaction effect since We the People 
has more of an effect in the AP classrooms than in the regular classrooms. 
Possibly AP students have more experience with cooperative group learning 
and the critical thinking skills required for this curriculum and thus respond 
more readily than students in regular classrooms. 

To determine whether We the People adds significantly to student performance 
over and above class placement, we performed a linear regression analysis 
using the six independent variables as predictors of student scale score. Those 
factors that made a difference in student performance (in order of the strength 
of effect) are: AP course enrollment, overall achievement in school, exposure 
to We the People, parent educational level, and ethnicity. The only factor that 
did not make a difference was student gender. 

Thus, the hypothesis that participating in We the People significantly adds to 
student performance over and above the role of class placement is confirmed. 
AP classes can be considered an indicator of high ability, so we can extrapolate 
our results to indicate that We the People contributes positively to student 
performance even after accounting for the effects of ability. 



63.5 



49.0 



34 - 1 32.3 
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Next Steps 

These findings are preliminary; the full evaluation study will be able to extend 
these analyses. The final knowledge test (Forms A and B) developed by WestEd 
is a reliable, valid test of student knowledge of the U.S. Constitution and U.S. 
history and government and a viable evaluation instrument on which to base 
further studies. 

Now that the pilot test is complete, we hope to conduct a full-scale study, 
scheduled for the 2005-06 school year. This study will be conducted in 
multiple states using a representative sample of students and schools in each 
state. We plan eventually to extend this study, to follow students as they leave 
secondary school and enter the workforce or pursue an advanced education, 
measuring their ongoing levels of political and civic participation. Such a 
study should provide the best possible measure of the impact of We the People 
on student learning and attitudes towards politics. 

Conclusion 

We the People is a well-established curriculum used in all states. Over the past 
17 years, some 26 million students have participated in this curriculum. 
Finding a valid measure to assess its influence on student learning and 
attitudes is important because of the need to discover curricula that can 
counter the growing apathy and lack of civic knowledge among youth in the 
United States. This pilot study found evidence that students participating in 
We the People: 

m Developed a greater sense of citizen responsibility and obligations to 
the community; 

■ Had stronger feelings of political efficacy; 

■ Scored higher on achievement tests of knowledge of U.S. government 
and civics; and 

■ Showed greater interest in politics and current events. 



More information on the pilot study is available from: 

MPR Associates, Inc. 

2150 Shattuck Ave., Ste. 800 
Berkeley, CA 94704 
(510) 849-4942 
www. mprinc. com 
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Technical Tables 

Internal Consistency 

Reliability refers to the ability of a questionnaire to measure attitudes 
accurately. Internal consistency reliability (Marshall and Hales 1972) is 
determined by checking the components of a questionnaire against each other. 
Internal consistency reliability in the form of Cronbach's Alpha is the type 
reported in this document. 

Internal consistency reliabilities vary from a low of 0.0 to a high of 1.0 and 
represent the proportion of the variance in the respondents' scores attributable 
to true differences on the psychological construct. DeVellis proposed the 
following: Alpha scores below 0.60 are unacceptable; from 0.60-0.64 are 
undesirable; from 0.65-0.69 are minimally acceptable; from 0.70-0.79 are 
respectable; and from 0.80-0.89 are very good. 7 At 0.90 and above, one should 
consider shortening the scale. 

Based upon these recommendations, the Cronbach's Alpha scores as shown in 
this table were obtained for the scales used in this report. All are acceptable, 
and only one score is "undesirable." Most fall in the respectable range. 



Number of 



Factor 


Alpha 


items 


Attention to politics 


0.82 


9 


Use of information 


0.74 


5 


Political participation 


0.69 


11 


Sense of political efficacy 


0.72 


6 


Responsibilities of a citizen 


0.77 


5 


Obligations of a citizen 


0.74 


7 


Sense of political and community responsibility 


0.83 


5 


Assessment of government's impact on their lives 


0.75 


3 


Evaluation of government 


0.73 


4 


Appreciation of democracy 


0.60 


4 


Political tolerance 


0.84 


6 


Tolerance of ambiguity 


0.65 


4 
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Regression Tables 

We used a full information maximum likelihood estimator: treating for the 
students' previous grades; whether at least one parent had attended college; 
whether the class was AP; the number of political, school, and community 
organizations they had joined prior to the beginning of the class; whether 
they speak a language other than English at home; and the reported level of 
political interest in other students at their school. We adjusted for clustering 
by teacher. 



Model 





1 


2 


3 


4 


Variable 


Attention to 
politics 


Use of 
information 


Political 

participation 


Sense of political 
efficacy 


Constant 


1.105*** 


0.603*** 


0.611 


-0.924*** 




0.159 


0.077 


0.042 


0.164 


Measure pre-test score 


0.486*** 


0.597 


0.611*** 


0.588*** 




0.029 


0.035 


0.042 


0.036 


Liked class 


0.171*** 


0.144*** 


0.028*** 


0.132*** 




0.027 


0.026 


0.005 


0.022 


African American 


-0.016 


0.108 


0.018 


0.085 




0.059 


0.065 


0.022 


0.083 


Latino 


-0.077 


0.005 


0.014 


0.027 




0.052 


0.046 


0.014 


0.059 


AP 


0.170 


0.145 


0.037 


0.194* 




0.118 


0.085 


0.039 


0.093 


WTP 


0.662*** 


0.540*** 


0.230*** 


0.599** 




0.144 


0.134 


0.044 


0.217 
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Model 




5 


6 


7 


8 


Variable 


Responsibilities 
of a citizen 


Obligations 
of a citizen 


Sense of political 
and community 
responsibility 


Assessment of 
government's 
impact on their 
lives 


Constant 


-0.977*** 


-0.706** 


0.380* 


-0.687* 




0.208 


0.228 


0.157 


0.304 


Measure pre-test score 


0.491*** 


0.589*** 


0.545*** 


0 379 *** 




0.042 


0.402 


0.029 


0.034 


Liked class 


0.103*** 


0.075* 


0.218*** 


0.123 




0.027 


0.034 


0.027 


0.065 


African American 


0.023 


-0.154 


0.136 


- 0.120 




0.089 


0.129 


0.082 


0.097 


Latino 


- 0.101 


-0.036 


-0.028 


-0.086 




0.076 


0.056 


0.050 


0.086 


AP 


0.128 


0.091 


0.119** 


0.154 




0.163 


0.128 


0.046 


0.094 


WTP 


0.933*** 


0.698*** 


0.521* 


0.386 




0.171 


0.187 


0.233 


0.281 



Model 




9 


10 


11 


12 


Variable 


Evaluation of 
government 


Appreciation of 
democracy 


Political 

tolerance 


Tolerance of 
ambiguity 


Constant 


-0.761* 


1.235*** 


-0.283 


-0.178 




0.329 


0.255 


0.154 


0.117 


Measure pre-test score 


0.426*** 


0.519*** 


0.560*** 


0.233*** 




0.039 


0.044 


0.027 


0.042 


Liked class 


0.103** 


- 0.011 


0.074** 


0.057** 




0.038 


0.036 


0.028 


0.019 


African American 


-0.160 


-0.062 


-0.028 


-0.134* 




0.096 


0.052 


0.096 


0.067 


Latino 


-0.071 


-0.081 


-0.096 


0.030 




0.075 


0.051 


0.060 


0.041 


AP 


0.015 


0.130** 


0.152** 


-0.048 




0.143 


0.047 


0.056 


0.032 


WTP 


0.704 


0.031 


-0.039 


0.041 




0.457 


0.432 


0.200 


0.130 
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